So, implementation-wise I think we might need to take a different approach. Right now, I see two main issues:

1. We're not running ExUnit.Formatters on dry runs—I think we might want to. Basically, the only thing we want --dry-run to do is to literally skip executing the body of tests (+ setup/setup_all calls and whatnot), but everything else would be the same.
2. We are printing test files, but this doesn't give us information about what tests (or test cases even, if there are >1 test cases in a file) would run. I think this is important. Think of using --dry-run to answer the question "if I pass this --exclude option, how many tests will be skipped"?.

Do these points make sense? I am not even sure this is very feasible, so we'd need to sort of proof-of-concept this.

I tried this locally and there are some issues.

The biggest one is that it prints Test dry run: for files that then exeute no tests. An example from stream_data:

→ mix test --dry-run
Running ExUnit with seed: 188086, max_cases: 28
Excluding tags: [:stdlib]

Test dry run:
Test dry run:
test/stream_data/lazy_tree_test.exs:16
test/stream_data/lazy_tree_test.exs:6
test/stream_data/lazy_tree_test.exs:77
test/stream_data/lazy_tree_test.exs:6
test/stream_data/lazy_tree_test.exs:8
test/stream_data/lazy_tree_test.exs:68
test/stream_data/lazy_tree_test.exs:52
test/stream_data/lazy_tree_test.exs:6
test/stream_data/lazy_tree_test.exs:6
test/stream_data/lazy_tree_test.exs:6
test/stream_data/lazy_tree_test.exs:35
Test dry run:
Test dry run:
Test dry run:
Test dry run:

I think we should print the Test dry run header only once and then just print all tests. Then, we should likely do some formatting. I’m thinking to use color in the header and call it something like Tests that would be executed:, in :cyan (respecting color settings for Mix).

Thoughts?

This looks good to me, just two tine additions to tests, but @whatyouhide should verify that it also works for his initial use case.

This is lovely 💯